convolution : son et lumière

Exterior night, cityscape. Thousands of distant points of light lie still in the summer air. Below them a neon diner sign flickers silently across a nearby empty street. Suddenly the close up face of the main character moves into the frame, the focus pulls, bringing their silent profile into crisp silhouette, while blurring the urban backdrop into a splash of color. Each point of light blooms into a hexagon, the neon sign now an unreadable splotch of red. The main character pauses for a second, then moves on into the night. Fade to black. Something special just happened in this imagined movie scene, when viewed through a scientific lens. The cityscape image, in its transition from crisp to out-of-focus, underwent a convolution, a process with a precise mathematical meaning that will become our main character in this short piece. I will try to explain as accurately as possible the concept of convolution, using familiar examples from our everyday lives. Convolution is in fact all around us—we just have to attune ourselves to it, to shift to a viewpoint in which lights and sounds and motions become patterns of numbers and signals. It could be argued that this type of shift, with the logical power to model and predict that it brings, is an idea that has opened the doors to centuries of tremendous human scientific and technological creation. I hope to leave you with a glimpse of how an engineer or scientist, or at least this applied mathematician, thinks about the world. The classic cinematic trope above, toying with depth of field, mirrors what our eyes and brain do daily, mostly unconsciously, as we shift attention in a complex three-dimensional visual world. The myopic among us may recreate the same blurring effect for free, simply by taking off our glasses and staring into the distance (here the hexagons are instead discs). Let’s examine more carefully what took place. We may treat the cityscape image as a pattern of light intensity existing across a two-dimensional plane, as in a rectangular photographic still, or a grid of pixels on your computer screen. Just like Manhattan’s grid of avenues and streets, each point in this plane has a location given by its horizontal distance from the left side (x coordinate) and vertical distance up from the bottom (y coordinate). At each such point the image has a brightness, which may treat as a number, zero for black, one for bright, two for very bright, . . . and so forth. So we have a number at each point in the plane: mathematically this